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BACKGROUND OF THE INVENTION 

The present invention relates to interactive hand-held 
20 devices, and more particularly to a method and apparatus for 
modulating video signals with auxiliary data for reception on and 
use by receivers such as hand-held devices, and providing 
promotional opportunities and other benefits to users of these 
receivers from the reception of the signals. 

25 
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Users of these hand-held devices and other receivers 
selectively receive auxiliary data for purposes including 
enjoyment, promotion, transfer of information, data collection, 
commercial verification, security, education, and transactions or 
5 verifications at points of sale, as well as other commercial, 
personal, entertainment, or amusement purposes collectively 
referred to herein as "promotional opportunities" . 

U.S. Patent 4,807,031 to Broughton et al . ("Broughton") 
10 entitled "Interactive Video Method and Apparatus" relates 

generally to in-band video broadcasting of commands and other 
encoded information to interactive devices. The invention 
described therein relates generally to interactive educational 
and entertainment systems, and is described in one embodiment in 
15 the context of television program control of toys located where 
there is a television receiver, as within a residence. 

To encode control data capable of providing a benefit to a 
user, Broughton discloses a novel method of luminance or 

2 0 chrominance modulation of a video signal that creates a composite 
video signal (i.e., a modulated video signal), wherein the video 
signal is modulated with control data. The novel modulation 
method alternately raises and lowers the luminance/chrominance of 
adjacent horizontal scan lines to create a video subcarrier that 

2 5 contains the control data. 
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In Broughton, the video signal is not being replaced with 
other data, nor is the data being added as a separate signal 
along with the video signal. Rather, the video signal itself is 
5 modulated to carry the control data. Therefore, the control data 
is a part of, or contained within, the video signal and yet is 
imperceptible to the human eye. The encoding method also 
includes preview and remove circuitry to ensure suitability or 
the presence of data encoding and removal of data encoding, 
10 respectively. 

The control data is transmitted either by television 
broadcast means, or by pre-recorded video players that are 
connected to a video display. The control data is then received 

15 by the video display where at least one video field of the video 
display is modulated by control data. The control data is then 
detected with either opto-electronic or radio frequency (RF) 
detection means that discriminate between the program material 
and the control data to detect the control data. The detected 

20 control data is further reproduced so that the control data can 
be used with an interactive device. 

The encoding method of Broughton may be used to record a "1" 
or a "0" on every field of a television video signal. Since 
2 5 under NTSC standard the video signal is broadcast at 3 0 frames a 
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second, there are 60 fields per second resulting in a data rate 
of 60 bits per second. However, it is often difficult to achieve 
60 bits per second of reliable continuous data using this method 
as will be explained in greater detail below. 

5 

In addition to Broughton's data transmission rate being 
relatively slow, it may not be reliably used for transmitting 
long data strings to portable devices and other receivers for 
processing and reproduction. Accordingly, Broughton in one 

10 application is typically used by interactive devices to capture 
auxiliary data that triggers action on the device (i.e., the 
device acts in a triggered mode) . Furthermore, the auxiliary 
data signals of Broughton are detected asynchronously, as the 
decoding process on the interactive devices and other receivers 

15 (e.g., decoder boxes) are not synchronized to the video signal 
received from the display device. Moreover, the beginning of a 
video field was undeterminable by the interactive device of 
Broughton. 

2 0 Broughton preferably operates by superimposing a 7875 Hz 

subcarrier on the video signal, which is at a rate of half the 
15750 Hz horizontal retrace frequency. To modulate the video 
signal with a subcarrier signal, the intensity of one horizontal 
line is raised and the intensity of the next line in a field is 

25 lowered thereby resulting in the 7875 Hz subcarrier signal. 
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An example of an implementation of Broughton is as follows: 
an encoder splits each field of a video signal into 8 equal 
5 slices, with each slice occupying approximately 2 milliseconds 
and encoded with the same data bit. The interactive device 
records and tracks every eighth bit, and thereafter compares the 
most recent 8 bits (i.e., 1 byte) it receives to the desired 
combination of 8 bits stored in a table on the interactive 

10 device. If there is a match between the two different sets of 8 
bits, the interactive device then proceeds to match a second byte 
and then a third byte. Once all three bytes are matched, action 
is triggered on the interactive device, which may include a 
visual or audio notification of a promotional opportunity. 

15 Despite the success of Broughton, there is a need in the art for 
a new apparatus and method for modulating a video signal with 
data that is faster and more reliable. 

Improvements on the method of modulation described in 
20 Broughton are described in U.S. Patent 6,094,228 to Ciardullo et 
al . and U.S. Patent 6,229,572 to Ciardullo et al . (referred to 
collectively herein as "Ciardullo" ) . Both Ciardullo patents 
describe improved methods of modulation wherein the carrier 
signals relating to control data (i.e., auxiliary data) are 
25 inserted on the visual portion of a video signal by changing the 
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luminance of paired lines in opposite directions. Instead of 
raising and lowering the intensity on an entire scan line of a 
video signal as in Broughton, Ciardullo uses pseudo noise 
sequences to raise and lower the intensity on portions on a 
5 series of first lines on every other video scan line in a field 
of a video signal, where the lines paired to the first lines are 
modulated with the inverse pseudo noise sequences. Ciardullo 
thereby allows larger amounts of auxiliary data to be modulated 
in video signals by use of the pseudo noise sequences. Broughton 
10 and Ciardullo, which are owned by the assignee of the present 
invention, are incorporated by reference herein. 



Prior efforts by the assignees of the present patent 
application include United States Utility Patent Application 

15 entitled ''Interactive Optical Cards and Other Hand-Held Devices 
with Increased Connectivity", Serial No.: US 09/489,373, filed 
January 21, 2000 of Edward J. Koplar and Daniel A. Ciardullo 
("Koplar I")/ which is incorporated by reference herein. Koplar 
I relates to various methods and apparatuses for use with 

20 promotion opportunities, such as interactive advertising and 
gaming. Koplar I describes various methods for receiving and 
providing data and signals to hand-held devices, as well as 
apparatuses for use with promotional opportunities and methods of 
using the same. 



#648315.05 
55121-88061 



Another patent application by the assignees of the present 
invention is United States Utility Patent Application entitled 
"Universal Methods and Device for Hand-Held Promotional 
Opportunities", Serial No.: 09/829,223, filed 9 April 2001 of 
5 Edward J. Koplar, Daniel A. Ciardullo, James G. Withers and 
Christopher E. Chupp ("Koplar II"), which is incorporated by 
reference herein. Koplar II describes additional methods for 
receiving and providing data and signals to hand-held devices, as 
well as apparatuses for receiving promotional opportunities and 
10 methods of using the same. 

Yet another patent application by the assignees of the 
present invention is U.S. Patent Application entitled "RBDS 
Method and Device for Processing Promotional Opportunities", 

15 Serial No. 10/126770, filed on April 19, 2002, of James G. 
Withers and Alan G. Maltagliati (referred to hereinafter as 
"Withers"), which is incorporated by reference herein. Withers I 
describes further improvements to Koplar I and Koplar II 
including the transmission of auxiliary data to a hand-held 

2 0 device by use of the RBDS system. 

For purposes of the present invention, the term "hand-held 
device" means an interactive device of portable character, 
preferably of hand-held type that may be carried in the palm by a 
25 user, between fingers of the user, or is otherwise intended to be 
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easily grasped and handled manually by the user. By way of 
example, hand-held devices includes smart cards, mobile phones, 
personal digital assistants (PDAs) , gaming devices and other 
hand-held devices capable of receiving and processing auxiliary 
5 data. 

The present invention need not be implemented by 
manufacturing a customized hand-held device to incorporate 
functionality of the present invention. Hand-held devices may 
10 have a slot that typically allows the device to receive memory 
cards and sometimes may allow it to use specially designed 
interface cards to receive other types of information. 



A memory card, which may also be referred to as a flash 
15 memory card or a storage card, is a small storage medium that 

typically uses flash memory to store data such as text, pictures, 
audio, and video for use by small, portable electronic and 
computing devices. Memory cards on the market as of the date of 
the present invention include the SD™ (secure digital) card, the 
2 0 CompactFlash® card, the Memory Stick® card, the Mult iMediaCard™ 
(MMC) and the SmartMedia® card. Memory cards are non- volatile 
solid-state devices that offer a combination of high storage 
capacity, fast data transfer, increased flexibility, excellent 
security and small size. Memory cards are typically accepted via 
25 a slot on portable devices. 
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Although slots are primarily used to receive memory cards 
that only provide data storage, some of the slots' functionality 
are not so limited and are capable of interfacing with peripheral 
5 devices. An example of the foregoing is the SD slot, which was 
originally intended to provide portable devices with flash 
memory. However, the slot was written with an open standard so 
that computer software operating on a portable device may be 
written to control peripheral devices connected through use of 
10 the SD slot, and such hand-held devices with a slot and open 

protocol are referred to herein as "slotted hand-held devices" . 
At the time of this invention, SD slots are available in cameras, 
cell phones, MP3 players and other portable devices. 

15 An interface device is a card capable of insertion into a 

card slot with a primary purpose of obtaining data or other input 
from sources not traditionally available to a slotted hand-held 
device and is referred to hereinafter as an "interface card" . 
The biggest advantage of manufacturing and using an interface 

20 card (i.e., with a slotted hand-held device as opposed to a 

customized hand-held device) is that the interface card takes 
advantage of the resources (e.g., an LCD, Internet access, a 
keypad, etc.) on existing slotted hand-held devices, thereby 
reducing the manufacturing cost and increasing the functionality 

25 of existing such devices. 
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An interface protocol such as SDIO (the SD input and output) 
protocol is a standard implemented on various slotted hand-held 
devices that allows storage media and peripheral devices to be 
5 operated through use of an appropriate slot (e.g., the SD slot) . 
For example, a PDA with a SD slot and enabled with the SDIO 
protocol and related software that has a SD card outfitted with a 
camera-like device inserted into its SD slot may use a camera- 
like device to take photographs that are stored on the PDA or are 
10 automatically uploaded to a predetermined Internet location. 

Another device that may be configured to be a slotted hand- 
held device is the Nintendo® Gameboy® game unit. As of late 2002, 
there are approximately 25,000 Nintendo® Game Boy Advance® game 

15 units sold every day, and over 100,000,000 Gameboy game units 
currently in use worldwide. While Gameboy game units are not 
equipped with a slot such as a SD memory slot, a company 
operating under the name X-traFun has developed a Bluetooth™- 
ready Gameboy cartridge outfitted with a SD slot. The cartridge 

2 0 provides a network mechanism to communicate, and the slot enables 
the reception of storage and interface cards to provide 
functionality of the present invention. 

The term "computer" is also used herein in its broadest 
25 possible sense, and may include without limitation a laptop, 
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compact or personal computer, mobile phone, gaming device, 
personal digital assistant (PDA) , or other computer-like device, 
or other devices using one or more microprocessors or digital 
processors to achieve a computing or data processing or data 
5 manipulative process or comparable or similar functions. 
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SUMMARY OF THE INVENTION 



The present invention comprises a method and device for 
modulating a video signal with data. More specifically, the 
method involves a user directing a slotted hand-held device with 
an interface card at a display device that is presenting video 
signals modulated with auxiliary data from a broadcast source. 

The hand-held device has a microcontroller and associated 
circuitry that determines the timing of the video signals by use 
of a vertical retrace signal of the display device. The video 
signals are received on the interface card by use of an optical 
detector. The microcontroller of the interface card then 
determines whether auxiliary data is present in the video 
signals. The microcontroller next evaluates whether the received 
auxiliary data relates to data packets. If so, then the 
microcontroller assembles the data packets to provide the user 
with useful data and providing the user of the slotted hand-held 
device with promotional opportunities or other benefits from the 
reception of useful data. 

In the preferred embodiment of the present invention, 
various suitable hand-held devices may be used to provide the 
functionality of the present invention by use of an interface 
card attached to a slotted hand-held device and appropriate 
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software running on the slotted hand-held device to interpret the 
received data. Preferably, the present invention uses a SD card 
and the SDIO protocol to interface with Nintendo's Gameboy, 
Compaq's iPAQ, and other slotted hand-held devices that include a 
5 SD slot. 



When auxiliary data is reproduced by use by the interface 
card or hand-held device, various signals, indications, display 
readouts, or other interactive events provide the user with 

10 promotional opportunities and other benefits according to content 
of the auxiliary data. The various interactive events described 
in Koplar I, Koplar II and Withers, incorporated by reference 
herein, are usable interchangeably by and in conjunction with the 
hand-held device and methods of use with the present invention. 

15 The interchangeability includes selective use of the features of 
the present invention, along with selective use of any of the 
various apparatuses and methods of Koplar I, Koplar II and 
Withers . 
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BRIEF DESCRIPTION OF THE DRAWINGS 



Fig. 1 is a flow chart of the method of encoding of the 
present invention . 

5 

Fig. 2 is a flow chart of the method of decoding of the 
present invention. 

Fig. 3 is a block diagram of the decoder of the present 
10 invention. 

Fig. 4 is a an alternate embodiment of the method of 
decoding of the present invention. 

15 Fig. 5 is a block diagram of the encoder of the present 

invention . 

Fig. 6 is a schematic circuit arrangement of one embodiment 
of the hand-held device of the present invention. 

20 

Fig. 7 is a schematic circuit diagram of an analog pre- 
filter of the data decoder of the invention. 
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Fig. 8 is a schematic circuit diagram of analog vertical 
synch and signal strength detection of the data decoder of the 
invention. 

5 Fig. 9 is a schematic circuit diagram of data signal 

detection circuitry of the data decoder of the invention. 

Fig. 10 is a schematic circuit diagram of a microcontroller 
of a first embodiment of the data decoder of the invention. 

10 

Fig. 11 is a schematic circuit diagram of a microcontroller 
of a second embodiment of the data decoder of the invention. 

Fig. 12 is a flow chart of the method of decoding of the 
15 present invention. 

Fig. 13 is a graphic representation of the fields of the 
present invention. 

20 Fig. 14 is a graphic representation of a data packet used in 

the present invention. 

Corresponding characters indicate corresponding elements in 
the views of the drawings. 

25 
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DETAILED DESCRIPTION OF THE INVENTION 



The following method and apparatus for a modulating a video 
signal with data is an improvement upon the method and apparatus 
5 previously disclosed in Broughton. Broughton discloses a 

communication system that allows auxiliary data to be received 
from a display device, wherein the data is preferably read 
optically by a hand-held device through use of a photodetector . 

10 Referring to Fig. 1, a video signal 18 is transmitted from a 

signal source 10 to an encoder 12. Video signal 18 is preferably 
an analog NTSC video signal, but may be other video signals or 
video signal formats compatible with the present invention. 
Signal source 10 is typically a professional grade video tape 

15 player with a video tape containing a video program, but may also 
be other sources of video signals including a camcorder or a 
digital versatile disc (DVD) player with a DVD video containing a 
program. Encoder 12 is described in further detail in the 
description of Fig. 5 below. 

20 

Operator 16 interacts with encoder 12 to control its 
operation. Preferably, operator 16 is a person that interacts 
with encoder 12 through the use of a computer or other electronic 
control device. However, operator 16 may consist entirely of a 
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computer or other electronic control device that directs 
operation of encoder 12 in an automated manner. 

A carrier signal 20 is selectively added to video signal 18 
5 by operator 16 at encoder 12 to modulate auxiliary data 21 within 
video signal 18. The method of adding carrier signal 20 is by 
selectively increasing and decreasing pixel intensity of paired 
scan lines as disclosed in Broughton. However, the present 
encoding method differs from Broughton in that encoder 12 splits 

10 each field of video signal 18 into multiple segments, such that 

each segment may be individually modulated with carrier signal 20 
and the complement of the data bit broadcast on the first field 
is modulated on the second field, as will be described in further 
detail below. It should be appreciated that the present method 

15 of selectively modulating can reverse the order of the fields 

(i.e., the first field and second field) as needed or desired in 
a particular embodiment. In the preferred embodiment, the 
present invention utilizes four equal segments per field. 

20 Upon modulating video signal 18, encoder 12 outputs a 

modulated video signal 22 comprised of video signal 18 and 
auxiliary data 21 (i.e., a composite video signal). Modulated 
video signal 22 is then provided to a broadcast source 14 for 
distribution to end-users who will view the program. Broadcast 

25 source 14 is preferably a television broadcast station that 
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broadcasts programs, but also may be other broadcast video 
sources and DVD media and other media sources including video 
tapes that will be provided to one or more end users. 

Referring to Figs. 2 and 3, broadcast source 14 provides 
modulated video signal 22 to display device 26. Display device 
26 is representative of a television screen, video monitor or 
other video display, movie screen, computer monitor, video- 
converted display or video-like display, capable of receiving 
analog or digital video or video-representative signals from a 
suitable signal source, such as a television transmitter, a 
videotape, a streaming video server, a DVD, or the computerized 
display representation of such a source of image content. For 
present purposes, it will be assumed that display device 26 is a 
kinescope or other conventional type of television display or 
monitor (which may include multiple or single-beam types of 
projector displays) . 

Display device 2 6 may schematically represent a video 
20 display for displaying video signals 18 but may also be any sort 
of electron gun, active, array or passive array display device 
capable of providing not only imaged information in a visible 
mode but also auxiliary information (e.g., data) in a 
substantially transparent mode. Display device 26 may be further 
25 characterized as a computer monitor or display, as well as a 



10 
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portion or computer window of display device 26. Display device 
26 may also be a high definition or digital television, or other 
digital video presentation device. Display device 26 may vary in 
size, and may be small like a Sony® Watchman®, or large like a 
5 movie screen or a Sony Jumbotron®. Video signals 18 receivable 
from broadcast source 14 by display device 2 6 include those 
delivered by microwave relay, satellite retransmission or cable, 
streaming and other types of downloadable or viewable computer 
video presentations, and those generally made available by wired 
10 or wireless methods. 

Modulated video signal 22 is presented on display device 26. 
Slotted hand-held device 2 9 optically receives modulated video 
signal 22 by use of a photodetector 62 on an interface card 60. 

15 Slotted hand-held device 2 9 may be in the form of any type of 
hand manipulable device such as a smart card, cell phone, PDA, 
game unit or other palm like device that has a slot 66. Slotted 
hand-held device 2 9 may be held in the palm or between the 
fingers of a user in the general vicinity of display device 26, 

20 typically within the same room and, when necessary, oriented so 
that photodetector 62 may optically receive light from a visual 
representation of modulated video signal 22 from display device 
26. It should be appreciated that photodetector 62, interface 
card 60, data decoder 72 and slotted hand-held device 2 9 may 
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alternatively be combined in a non-slotted hand-held device 28 
such as used in Koplar I . 

In the preferred embodiment of the present invention, a 
5 slotted hand-held device 29 such as a PDA, game unit, or cellular 
telephone is outfitted with a slot 66 and contains software 
and/or other electronics to run an interface protocol 68. The 
present invention may be used with various interface cards 60 and 
slots 66 provided that device 29 uses an interface protocol 68 

10 that allows for expansive use of its protocol (i.e., an "open 

protocol") with card 60 and slot 66. Preferably, slot 66 is a SD 
slot and interface protocol 68 is the SDIO protocol. An 
interface card 60 also operating interface protocol 68 is 
inserted into slotted hand-held device 29, and device 29 runs 

15 software to interpret data received by card 60 and passes it to 
device 29 by use of interface protocol 68. 

As further shown in Fig. 3 modulated video signal 22, 
comprised of video signal 20 and auxiliary data 21, is 

20 transmitted from display device 26 and detected by photodetector 
62 on interface card 60. Thereafter, the received modulated 
video signal 22 is decoded by a data decoder 72 located on 
interface card 60. The decoded auxiliary data 21 is passed to 
slotted hand-held device 29 through use of an interface protocol 

25 68. Slotted hand-held device 29 utilizes auxiliary data 21 to 
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provide the user with a benefit or promotional opportunities 
based on the receipt of auxiliary data 21. The promotional 
opportunities may be redeemed or obtained using the optional 
wireless Internet access 70, as described in Koplar I. 

The slotted hand-held device 2 9 preferably uses wireless 
Internet access 7 0 to provide and/or redeem promotional 
opportunities over the Internet. The preferred protocol for 
wireless Internet access is the use of Bluetooth. Bluetooth 
provides a means for RF data to be passed in a specified form at 
a 2.4 GHz frequency between slotted hand-held device 29 and a 
transceiver that is connected to the Internet. It will be 
appreciated in the art of Internet networking that other means of 
providing wired and wireless Internet may be used with the 
present invention including Wifi 802.11. 

As a further embodiment of the present invention, interface 
card 60 may, in addition to or an alternative of photodetector 
62, receive modulated video signal 22 or auxiliary data 21 by 
other means including by use of a RF receiver such as when 
broadcast source 14 is a decoder box that demodulates the 
modulated video signals and transmits auxiliary data 21 by 
various methods known in the art of signal transmission including 
RF. 
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Also, when desirable, data storage (not shown) may be added 
to interface card 60 to store auxiliary data 21 or promotional 
opportunities for later recall. It should be appreciated that, 
depending on the application, it may be desirable to provide a 
large amount of space on interface card 60 thereby allowing a 
significant amount of data to be stored and later retrieved. 

In an alternate version of the optical detection of Fig. 2, 
Fig. 4 shows electrical detection of auxiliary data 21. 
Broadcast source 14 provides modulated video signal 22 to a 
decoder 13. As discussed in greater detail below, decoder 13 
acts in a similar manner to data decoder 72 by determining 
whether auxiliary data 21 is present in modulated video signal 22 
but also contains transmission device 24 and a video output to 
provide the modulated video signal 22 to a display device 26. 
Auxiliary data 21 is provided to a transmission device 24, which 
transfers the information to a non-slotted hand-held device 28 
without a photodetector 62 but with a non-optical receiver 64 
such as RF receivers, infrared and computer-like 

interconnections. Modulated video signal 22 is passed unaltered 
through decoder 13 and out the video output and presented on 
display device 2 6 so that a user may watch the video program. 
Upon receipt of auxiliary data 21, non-slotted hand-held device 
2 8 provides the user with a benefit or promotional opportunity. 
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In an alternate version of the foregoing, non-optical 
receiver 64 may be implemented on interface card 60 such that 
slotted hand-held device 29 replaces non-slotted hand-held device 
28 and acts in a manner consistent with the foregoing 
5 description. 

Encoder 12 in Fig. 5 includes a digital video input 30 to 
receive a video signal 18 from signal source 10 and to pass it to 
micro-controller 36. However, encoder 12 may receive an analog 
10 video signal 18 via analog video input 32 and analog to digital 
converter 34. Analog to digital converter 34 digitizes the 
analog video signal 18 according to known techniques such that it 
may be sent to micro-controller 36. 



15 Micro-controller 36 is electronically connected to a carrier 

presence 38, which provides micro-controller 36 with the timing 
of where, when and at what intensity encoder 12 should insert 
carrier signal 20 into video signal 18 at the direction of 
operator 16. Preferably, such instructions are received from 

20 operator 16 by carrier presence 38 via a serial port. However it 
should appreciated in the art of electronics that other device 
interconnects of encoder 12 are contemplated, including via 
universal serial bus (USB), "Firewire" protocol (IEEE 1394), and 
various wireless protocols. In an alternate embodiment, carrier 
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presence 38 may be an operator interface so that operator 16 can 
directly interface with encoder 12. 

Once micro-controller 36 receives video signal 18 and 
5 information from carrier presence 38, software 50 manages further 
operation of encoder 12 and directs micro-controller 36 to store 
the chrominance information of video signal 18 in storage 40. 
Encoder electronics 42 at the direction of software 50 preferably 
uses the methods of Broughton and the present invention as will 

10 be described in further detail below to modulate carrier signal 
20 into the luminance of video signal 18 thereby creating 
modulated video signal 22. The resulting modulated video signal 
22 is then sent digitally from encoder 12 by digital video output 
44, or in analog form by converting the resulting digital signal 

15 with digital to analog converter 46 and outputting modulated 
video signal 22 by analog video output 48. 

Micro-controller 36 may consist of more than one processor 
to manage the various processing and input /output of the present 

20 invention, but preferably consists of a single processor. 

Moreover, the specific electronics and software used by encoder 
12 may differ when its technology is included in a pre-existing 
device as opposed to a stand alone device. Encoder 12 may 
comprise varying degrees of hardware and software, as various 

25 components may interchangeably be used as either. 
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Fig. 6 shows an overview of the preferred embodiment of a 
microcontroller 100 and device circuitry 102 of data decoder 32. 
Device circuitry 102 is comprised of an analog pre-filter 104, a 
vertical detect/signal strength circuitry 106 and an auxiliary 
data detector 108, all of which are operatively associated with 
each other and microcontroller 100 as shown. 

Referring to Fig. 7, the preferred embodiment of analog pre- 
filter 104 that prepares the circuit for accurate digitization 
first comprises a current voltage converter 300. Current voltage 
converter 3 00 first comprises a photodetector D200, which 
optically reads a video signal 18 from display device 26 and 
outputs a current to voltage converter 300. Thereafter, current 
voltage converter 300 transforms the current detected by 
photodetector D200 into voltage. The circuitry of voltage 
converter 300 further comprises three resistors, R200, R201 and 
R202, three capacitors C200, C201 and C202, an operational 
amplifier U200 and voltages VA and VCC, all of which are 
operatively associated as shown. 

An automatic gain control 310 portion of analog prefilter 
104 amplifies video signal 18 by changing the resistance on the 
feedback circuit. The amount of gain provided to the circuit is 
controlled by microcontroller 100. Automatic gain control 310 is 
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used with interface card 60 as the distance and intensities 
received from display device 26 will vary. Accordingly, when the 
strength of video signal 18 is low, it is desirable to add gain 
so that a better reading of video signal 18 is possible. 
5 Therefore, the present invention measures the signal strength and 
decides whether to lower or increase the gain. The components of 
automatic gain control 310 include resistors R203, R208 and R209, 
capacitor C212, voltages VA and VCC an operational amplifier 
U400C and a digital potentiometer U204, all of which are 
10 operatively associated as shown. 

Video signal 18 passes from automatic gain control 310 to 
low pass filter - 80 KHz cutoff 320. This circuit provides a low 
pass filter that removes the high frequency noise from the signal 
15 by eliminating all frequencies above a preset level (i.e., 80 
kilohertz) . The components of the low pass filter - 80 KHz 
cutoff 320 include resistors R210, R204 and R205, capacitors 
C203, C204, and C206, voltages VA and VCC and operational 
amplifier U2 01, all of which are operatively associated as shown. 

20 

Video signal 18 then passes from the low pass filter - 80 
KHz cutoff 320 to the high pass filter 7 KHz cutoff 330. High 
pass filter 7 KHz cutoff 330 cleans the signal below 7 KHZ by 
discarding the undesired signal. The components of the high pass 
25 filter - 7 KHz cutoff 330 include resistors R206 and R207, 
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capacitors C207, C208, C209, C210 and C211, voltages VA and VCC 
and operational amplifier U201, all of which are operatively 
associated as shown. Once the signal passes through high pass 
filter 7 KHz cutoff 330, the pre-f iltering of video signal 18 is 
5 complete. 

Referring now to FIG. 8, the circuitry of vertical 
detect/signal strength circuitry 106 in the preferred embodiment 
comprises a signal strength detector 34 0 and an analog vertical 

10 sync 350. Signal strength detector 340 first comprises a 

rectifier D300 that polarizes video signal 18. Thereafter, video 
signal 18 transitions through a buffer comprised of resistor R306 
and R307, voltage VA, and operational amplifier U300B to invert 
the signal. The last portion of this circuit is an integrator, 

15 which measures the strength of video signal 18 and is comprised 
of resistor R308, capacitor R305, voltage VA and operational 
amplifier U300C, all of which are operatively associated as 
shown. The output of the integrator is signal SSRES1 and is 
received by microcontroller 100 which then resets the signal 

2 0 strength integrator. 

The signal for analog vertical sync 3 50 is passed from low 
pass filter - 80 KHz cutoff 320. Analog vertical sync 350 
generates the desired vertical synchronization signal used to 
25 synchronize the reading of data bits from auxiliary data 21. The 
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first part of circuitry is a gaining amplifier that gains the 
signal and inverts it by use of resistor R300 and R301, capacitor 
C3 00, voltage VA and gate U300D. The second part of the 
circuitry provides a small filtering stage. Resistors R302 and 
5 R303 and capacitor C301 filter high spikes out of the signal. 

Thereafter, the signal is processed by schmitt trigger 
inverters U302A and U302D, the double inversion acting as a 
buffer. Because of the nature of the schmitt trigger, voltage 

10 spikes are removed from the signal. Thereafter, the signal 

triggers a flip flop 74HC74 . The output of the flip flop 74HC74 
triggers a 555 timer 555, which is used to generate a constant 
timed pulse. During each pulse, regardless of how many triggers 
the 555 timer 555 receives, it will not generate interloping 

15 pulses until it times out. Resistor R305 may be used to adjust 
the time constant, by which the pulse can become wider or 
narrower. If microcontroller 100 detects the pulse and then 
waits for another vertical retrace period (i.e., 16.67 
milliseconds) and detects the pulse again, it knows that it has 

20 locked on the vertical synchronization signal. 

Referring to FIG. 9, the pre-filter signal is passed to a 
horizontal notch filter 360 that removes video signal 18 at the 
horizontal line scanning rate (i.e., 15570 KHz) from modulated 
25 video signal 22 so that it does not interfere with the reading of 



#648315.05 
55121-88061 



auxiliary data 21. Thereafter, low pass filter with cutoff at 
VEIL Freq. 370 is used for extra filtering to ensure that the 
signal higher than the 8 kilohertz is discarded. Then, the 
signal is passed to a band pass filter 380 which is another stage 
5 of filtering around 8 kilohertz. The signal then goes through a 
signal rectifier 390 that acts as an integrator that has a gain 
of one. Finally, the signal travels through a VEIL signal energy 
integrator 400 that measures the strength of the signal by 
measuring its voltage. The aforementioned circuits consists of a 
10 number of resistors R400-R413, capacitors C400-410, voltages VA 
and VCC, and operational amplifiers U400A, U400B, U400D, U401D, 
U401C, and U401B, all of which are operatively associated as 
shown. 

15 As shown in FIG. 10, microcontroller 100 and associated 

circuitry as would be used in a first version of the combination 
of slotted-hand device 29, data decoder 72 and interface card 60 
in a non-slotted hand-held device 28 with photodetector 62 first 
comprises a power source consisting of four batteries BC500, 

20 BC501, BC502 and BC503 . Each battery is 1.5 volts, and the power 
is provided to voltage regulator U501. Voltage regulator U501 
provides microcontroller 100 with a steady 5 volts of power 
through D501. VCC and VA are also provided by voltage regulator 
U501, wherein by means of a voltage divider VA is 2^ volts and 
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VCC is 5 volts. VCC is being fed through a diode D501 that 
provides battery voltage. 

Diode D501 allows microcontroller 100 to go into a sleep 
5 mode so as to reduce its need for voltage. When microcontroller 
100 wants to sleep, it can shut of voltage regulator U501 and go 
into sleep mode. Dual analog switch MAX323 is used to reset both 
the signal strength integrator and the VEIL signal strength 
integrator via the microcontroller 100. 

10 

Further components present in this embodiment are speaker 
SPK1, transistor Q504, visual display U503, switches S500, S501, 
S502, S503 and S504, and interface RS232, all of which are 
operatively associated as shown and the use of which are in 
15 accordance with the present invention as well as the hand-held 
devices described in Koplar I, Koplar II and Withers. 

In an alternate but preferred version of the present 
invention, microcontroller 100 in Fig. 11 is shown as implemented 

20 on interface card 60. Microcontroller 100 is shown in this 
preferred embodiment to be comprised of two separate 
microcontrollers U500 and U503, of which microcontroller U500 
controls the operations of microcontroller 100 as described in 
Fig. 10 above such as optical detection and determining whether 

25 auxiliary data 21 is present in video signal 18, and 
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microcontroller U503 manages the interface with slot 66 (i.e., 
SDIO connector J501) . SDIO connector J501 communicates with 
interface card 60 as described below. Microcontroller U503 
further controls the communication between the optical and SD 
5 portions of the circuitry. Preferably, microcontroller U503 runs 
at 8 MHz while microcontroller U500 runs at 4 MHz. 

Port RS232 is a computer interface which may optionally be 
included in various embodiments such as to debug the system to 
10 ensure that the proper auxiliary data is being received and 

modulated on interface card 60. Integrated circuit Max232 allows 
interface card 60 to communicate via a computer port RS232 so 
that interface card 60 may directly interact with a computer such 
as for debugging purposes . 

15 

In the preferred embodiment, a power source is not needed on 
interface card 60 as power is provided through slot 66. However, 
a charge pump U504 changes the three volts received from slot 66 
into five volts so as to properly power interface card 60. Other 
2 0 components as used in Fig. 10 as shown in Fig. 11. 

The preferred embodiment of a method of modulating a video 
signal 18 with auxiliary data 21 in FIG. 12. In a first step 
1000, a user orients or connects a receiver, such as slotted 
25 hand-held device 29, towards or in connection with a display 
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device 26 for the purpose of capturing modulated video signals 
28. Thereafter, in a second step 1100, a video signal 18 is 
captured from display device 26 by a photodetector 3 0 of 
interface card 60. A microcontroller 100 and device circuitry 
5 104 amplify, filter and shape video signal 18 received during a 
third step 1200. 

Still referring to Fig. 12, during a fourth step 1300 
microcontroller 100 determines whether auxiliary data 21 is 

10 present in video signal 18 (i.e., whether video signal 18 is 
modulated video signal 22) . If there is no auxiliary data 21 
present in video signal 18, the method returns to second step 
1100 and again attempts to capture modulated video signal 22. If 
in fourth step 1300 microcontroller 100 determines that auxiliary 

15 data 21 is present, then the method proceeds to a fifth step 1400 
wherein microcontroller 100 determines whether a complete data 
packet 112 has been received by interface card 60. If auxiliary 
data 21 is not in the form of data packet 112 or is otherwise 
unusable, auxiliary data 21 is discarded and the method returns 

20 to second step 1100. 

If auxiliary data 21 is in the form of data packet 112, 
during a sixth step 1600 microcontroller 100 determines the total 
number of packets 114 that interface card 60 expects to receive 
25 and the identification of a data packet number 116 of the just 
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received data packet 112. If the total number of packets 114 to 
be received by interface card 60 is 1, then microcontroller 100 
takes further action necessary to provide the user of slotted 
hand-held device 29 with a benefit as a result of receiving all 
5 of the desired auxiliary data 21 (e.g., promotional 

opportunities) as disclosed in a ninth step 1800. If total 
number of packets 114 is greater than 1, then microcontroller 100 
proceeds to an eighth step 1700 whereby microcontroller 100 
determines whether every data packet 112 corresponding to each 
10 data packet number 116 has been captured. 

If in the eighth step 1700 all data packets 112 have not 
been captured, interface card 60 returns to the second step 1100 
so as to attempt to capture the missing data packets 112. If all 

15 data packets 112 have been captured, slotted hand-held device 29 
proceeds to provide the promotional opportunities to the user 
according to ninth step 1800. During the ninth step 1800, a user 
of slotted hand-held device 2 9 receives promotional opportunities 
or other benefits that may include textual information, prizes, 

20 coupons, games, special access privileges, etc. Thereafter, the 
method of the present invention is terminated during a final step 
1900. 

During the foregoing method, the fields of video signal 2 0 
25 are encoded differently than in Broughton. Broughton's method 
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involves encoding each portion of a field 200 with an identical 
bit. As shown in FIG. 13, every slice 200a-h of field 200 
contains the bit u l" . 

5 With the present invention, fields 210a and 210b of a frame 

220 are encoded with complementary bits. Thus, a logic u l" is 
encoded in two fields 210a and 210b as "1 0", and a logic "0" is 
encoded in two fields 210a and 210b as "0 1" . When interface 
card 60 receives the bits, it performs a field comparison by 
10 subtracting the result of the intensity of two successive fields 
210a and 210b. This method produces a reliable data rate of 30 
bits per second. 

As shown in Fig. 13, the speed of the present invention is 
15 increased by splitting every field 210 into multiple segments 
with each portion having its own respective bit. Preferably, 
each field 210 is split into four equal portions such that each 
portions is at an equal offset from one another. Thus, fields 
210a and 210b splits appear in Fig. 13 with the 4 data bits 
20 decoded as "1 0 11". This method of splitting the fields 
produces a reliable rate of 120 bits per second data. 

For the foregoing method to function, during the second step 
of Fig. 12, microcontroller 100 needs to determine where fields 
25 210a and 210b begin during decoding so that it can properly 
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obtain all of the bits. In a first embodiment, data decoder 32 
obtains its proper timing by looking for and synchronizing to the 
vertical retrace period in video signal 20. 

5 Referring to Fig. 14, microcontroller 100 first looks for a 

section of the picture presented on display device 2 6 that is 
completely black (i.e., no video) and therefore may be a vertical 
retrace signal indicating that a vertical refresh of the picture 
has occurred. Thereafter, microcontroller 100 waits a sufficient 

10 time for another vertical refresh to occur (i.e., 16.67 

milliseconds) . If microcontroller 100 reads two successive 
regions that are completely black, and they are 16.67 
milliseconds apart, microcontroller 100 acts under the belief 
that it has locked on the vertical retrace signal and continues 

15 to read and record data packets 112 at a preferred displacement 
of 2 milliseconds from the beginning of the vertical retrace 
signal. If during the foregoing synchronization process data 
decoder 32 fails to detect a first and a second black region, it 
will continue attempting to synchronize by searching for black 

20 regions that are 16.67 milliseconds apart beyond where it 
previously looked. 

Once the vertical retrace signal is synchronized, and data 
decoder 32 successfully reads valid data, data decoder 32 is then 
2 5 in synchronization with video signal 18 and temporarily stops 
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looking for the vertical retrace signal. Under the first 
preferred embodiment of video signal 18 locking of the present 
invention, data decoder 32 locks on the vertical retrace signal 
for a few seconds and thereafter releases it and the 
synchronization process starts over again. Re-detection and re- 
synchronization of the present invention in this embodiment is 
preferred because timing of microcontroller 100 is not entirely 
accurate, therefore causing the synchronization of the vertical 
retrace signal to drift after a few seconds and making it more 
difficult to detect valid auxiliary data 21. 

In a second but preferred embodiment, the beginning of 
auxiliary data 21 is determined by looking for a sync. Data 
decoder 32 looks for a first data bit in data packet 112, such as 
A5, that acts as a marker or preamble. Data decoder 32 drifts 
very slowly across display device 2 6 trying to capture the 
marker, then data decoder 32 acts as though it is at the start of 
data packet 112 and sequentially reads the various bits. The 
data bits are read sequentially and then shifted in a register to 
its left, so that data decoder 32 receives the A5 byte first. 
Once a valid data packet 112 is received, then data decoder 32 is 
perfectly synced and can continue to read successive data packets 
112. Data decoder 32 is ensured of a valid data packet 112 by 
checking the CRC byte of data packet 112. 
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Auxiliary data 21 is read by microcontroller 32 in data 
packets 112 of preferably 8 bytes (i.e., 64 bits) in length. As 
shown in Fig. 14, the preferred embodiment of data packet 112 
first comprises a first byte 250 containing a preamble that 
5 identifies it as the start of data packet 112. A second byte 252 
contains the data packet number 26 relative to the total number 
of packets sent 25 (i.e., packet is 2 of 5) . The following five 
bytes 254, 256, 258, 260 and 262 (i.e., bytes 3-7) contain actual 
data, which may be compressed as will be readily understood by 
10 someone skilled in the art of data compression. The final byte 
264 contains a Cyclic Redundancy Check (CRC) to ensure that the 
received data bytes 250-262 were precisely matched with those 
transmitted from the signal source. 

15 Data packets 112 of a message 118 are typically sent by 

broadcast source 14 more than one time as will be appreciated in 
the art of computer networking. Message 118 in the preferred 
embodiment may contain up to 16 data packets 112. Data packets 
112 may be received by slotted hand-held device 29 or other 

2 0 receiver in any order, and any data packets 112 that have 

previously been successfully captured by slotted hand-held device 
29 will be ignored. Once all data packets 112 have been 
correctly received by slotted hand-held device 29, the entire 
message 118 is then declared to have been successfully decoded, 

25 and slotted hand-held device 2 9 takes appropriate action by 



#648315.05 
55121-88061 



providing the user with a benefit or one or more other 
promotional opportunities . 

It should be understood from the foregoing that, while 
5 particular embodiments of the invention have been illustrated and 
described, various modifications can be made thereto without 
departing from the spirit and scope of the invention. Therefore, 
it is not intended that the invention be limited by the 
specification; instead, the scope of the present invention is 
10 intended to be limited only by the appended claims. 
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